TV video is composed of 30 or more still images per second, referred to as frames. There is a problem in that each frame, whether digital or analog, includes degraded information, such as optical blurring or unsharpness, even if it is not blurred to such an extent that it becomes unclear.
FIG. 1 shows an example of degraded information included in a frame of actual TV video. FIG. 1 includes two images: the left image represents a frame composed only of Y (luminance components) of TV video acquired by using an X-ray pinhole camera; on the other hand, the right image represents an image in which degraded information has been reduced by subjecting the left image in FIG. 1 to super-resolution processing using related art invented by the inventor of the present invention (Patent Literatures 1 and 2). A comparison between the images in FIG. 1 indicates that actual TV video includes degraded information, such as optical blurring or unsharpness.
In the image restoration technologies invented by the inventor of the present invention (Patent Literatures 1 and 2), while repeating iterations according to formulas based on Bayse probability theory by using information about one still image including degraded information, such as optical blurring or unsharpness, a maximum-likelihood degradation factor and a maximum-likelihood restored image, i.e., an image after super-resolution processing, having a maximum likelihood for the luminance distribution of the still image are obtained through numerical computations. However, since a huge amount of computational processing is required for this calculation, there has been a problem in that it is difficult to handle TV video, which requires real-time processing.
There has been a problem with the image restoration technologies invented by the inventor of the present invention (Patent Literatures 1 and 2) in that they require a large scale of computation and are not suitable for substantially real-time processing since complex numbers are handled in the computation. In order to solve this problem, in a technology for which a patent application has been filed, the inventor of the present invention has changed the type of numbers handled in the computation from complex numbers to real numbers and proposed a hardware implementation using FPGAs (Field Programmable Gate Arrays), allowing substantially real-time processing. However, there have been problems in that the number of gates in the hardware implementation is as large as 1.5 million gates, and thus, costs are high and the installation area is large.
Another type of conventional super-resolution technology for TV video is a “super-resolution reconstruction” method (Patent Literatures 3 and 4), in which attention is paid to a certain object existing in a plurality of frames constituting TV video and the positions of that object are aligned to superimpose the plurality of frames, thereby realizing super resolution, which is a method that has been introduced into products. However, in a case where the size of the object considerably varies or where such an object is not included in a plurality of frames, such as in a scene involving intense motion or in a scene involving frequent zoom-ins or zoom-outs, there has been a problem in that super-resolution based on “super-resolution reconstruction” methods is not possible.
As another method, in a method described in Non-Patent Literature 1, Bayes statistical processing is executed on the basis of a plurality of successive still images acquired by using video cameras from mutually slightly different viewpoints, thereby obtaining a super-resolution still image. However, this method requires a large amount of memory for constantly storing a plurality of still images including degraded information. Furthermore, in order to obtain a still image having super-resolution, it is necessary to constantly process a plurality of still images. This requires a huge amount of computation as well as requiring a large amount of memory, which prohibits processing of TV video.